CAZON 
Roce! 


200 S 
TE6Z. Legislative Assemblée 
Assembly législative 
of Ontario de l'Ontario 





STANDING COMMITTEE ON 
PUBLIC ACCOUNTS 


TRAINING DIVISION 
(Section 3.11, 2002 Annual Report of the Provincial Auditor) 


Ath Session, 37th Parliament 
52 Elizabeth II 


': 


Digitized by the Internet Archive 
in 2022 with funding from 
University of Toronto 


https://archive.org/details/31/61114681984 


STANDING COMMITTEE ON PUBLIC ACCOUNTS 
MEMBERSHIP LIST 


4 SESSION 37" PARLIAMENT 


JOHN GERRETSEN 
Chair 
BRUCE CROZIER 
Vice-Chair 
GARFIELD DUNLOP RAMINDER GILL 
JOHN HASTINGS : SHELLEY MARTEL 
AL MCDONALD RICHARD PATTEN 


Anne Stokes 
Clerk of the Committee 


Elaine Campbell 
Research Officer 





Sy¥otr ls F 
Sainte JS = 





CONTENTS 


PREAMBLE 
Ministry Response to Committee Report 


1. OVERVIEW 


2. AUDIT OBJECTIVES AND SCOPE 
2.1 Overall Audit Conclusions 


3. JOB CONNECT AND SUMMER JOBS SERVICE PROGRAMS 
3.1 Measuring and Reporting on Program Effectiveness 
3.2 Monitoring Compliance with Program Guidelines 


4. APPRENTICESHIP PROGRAM 

4.1 Measuring Program Effectiveness 

4.2 Increasing Opportunities for Apprenticeships 

4.3 Updating Apprenticeship Standards 

4.4 Monitoring Program Quality and Compliance 

4.5 Enforcement of Legislation on Restricted Trades 

4.6 Effectively Assessing Prior Learning and Academic Readiness 
4.6.1 Prior Learning 
4.6.2 Academic Readiness 

4.7 Managing Program Funding for In-School Training 


5. LITERACY AND BASIC SKILLS PROGRAM 
5.1 Tracking and Reporting Participant Outcomes 
5.2 Linking Funding to Performance 
5.3 Monitoring Delivery Agency Performance 


6. ACQUISITION AND MANAGEMENT OF CONSULTING AND OTHER SERVICES 
6.1 Information Systems Development Projects 
6.1.1 Job Connect 
6.1.2 Literacy and Basic Skills 
6.2 GST Payments 


7. ADDITIONAL POINTS OF DiSCUSSION 

7.1 Labour Market Development Agreement 

7.2 Foreign-Trained Professionals and Tradespeople 
7.2.1 Availability of Information 
7.2.2. Barriers to Employment 

7.3 Local Training Boards 

7.4 Double Cohort 

7.5 Northern Ontario Medical School 

7.6,E.S.L. Funding 


8. LIST OF COMMITTEE RECOMMENDATIONS 
NOTES 


ODOWDONNN OOP BWW NNDM =| = = 


> 2s 19 


oe” 


F 





Beunnvipeey rrmeNn paricesoM he | 
oti inecineh pA ateaehayO gnizsadnl S& Be the 
7 cones it ith, aan aie aA i Ee ‘oP 






















sonéaornay fh ene Regen gotten & 4 <a 

eebert teria tos Rapelhigad tg Meartgowind 26° 6 9” 

ambos oir bie yin met int cerkbeek.» “evens ob .. ae 
7 . DAT itrin: toy t 7h, 7 pac 7a 

esrus ve 65% rE ww B. . ; > 


- 
GHIA) InOroc-1 10) Gron eiposS pripenaiy yay 9 


fas ra SonS | ine eA OMA ‘OARS. a 4 
pemediuG thaguinad Bathoge? dca gnlomt he oe ae 
Sermo! 4s Gs Tapia Se ee : 
A-wesnitty e¥ = nec Cuanied ore na Be 


eee 
_ 


7 


pupae! Wart) Oe, SAT sLcaS 4 feeeeal! Gay Lore oA..S 
aces) SNQaIeves 2 acral Gorsimnetal f Boi 2 
Dery a) yh, P ¢ 
ORE e283 wig aks $F a! 
es etierrye ‘ (ae) a 4 


AOA) 89 Swine. 7 
~s est, nee ae! the ote) tX- # 
mA Saban) (DoT, conmine2 25 BetigtT: ngito ¢T a = 
; hte 6 YSisaliewA 2 oe vite ae ; 
Mmornyatgn haat sain 6.4.7 _ 
Atm iasl Qala T emicy 

thoi) eehiecd § ¥ 

Lagu amalbens rene avenntr 
: shoal J 70,7: 4 








PREAMBLE 


The Provincial Auditor reported on the Training Division in Section 3.11 of his 
2002 Annual Report. The Standing Committee on Public Accounts held hearings on 
this audit report on February 12, 2003, with representation from the Ministry of 
Training, Colleges and Universities. This report constitutes the Committee’ s 
findings and recommendations. 


The Ministry supported the Auditor’s recommendations and presented the 
Committee with its plan for implementing them. 


The Committee extends its appreciation to the officials from the Ministry for their 
attendance at the hearings. Furthermore, the Committee acknowledges the 
assistance provided during the hearings by the Office of the Provincial Auditor, the 
Clerk of the Committee and staff of the Legislative Library’s Research and 
Information Services. 


Ministry Response to Committee Report 


The Standing Committee on Public Accounts endorsed the Provincial Auditor’s 
2002 report on the Training Division and recommended the implementation of his - 
recommendations by the Ministry of Training, Colleges and Universities. 


The Standing Committee on Public Accounts has prepared supplementary 
recommendations in this report, based on its findings during the hearings. The 
Committee requests that the Ministry of Training, Colleges and Universities provide 
the Committee Clerk with a comprehensive written response within one hundred 
and twenty calendar days of the date of tabling with the Speaker of the Legislative 
Assembly of Ontario, unless otherwise specified in a recommendation. In the event 
that the Ministry requires additional time to prepare a response to this report, it 
would be appreciated if the Deputy Minister's office contacted the Committee Clerk 
to determine a revised deadline. 


1. OVERVIEW 


The Training Division of the Ministry of Training, Colleges and Universities sets 
standards for employment services and adult literacy, helps employers develop a 
skilled workforce, and provides leadership on labour market and training issues. Its 
programs and services assist both individuals and employers in increasing skill 
levels, and help individuals make the transition from unemployment to 
employment, and from education and training to the labour force.’ 


The Division’s budget for 2002/03 was $386 million. That amount includes the 
federal government’s contribution of about $30 million to apprenticeship training. 
(This contribution is dependent on initial school registrations in a year.) The 
Division’s programs and services serve approximately 715,000 individuals and 
84,000 employers.” 


Division expenditures are distributed across the following areas: 


e Job Connect which helps youth get into the labour market; 
e Summer Jobs Service which assists youth with getting summer jobs; 
¢ apprenticeship training and related services; 


e Literacy and Basic Skills for adults aged 19 and over, out of school and in need 
of assistance; and 


e other programs and activities, including assistance for internationally trained 
individuals seeking to enter and practice a regulated profession in Ontario, and 
TVOntario with respect to distance education and training matters. 


Community colleges, school boards, and community-based, not-for-profit 
organizations are responsible for delivering Job Connect, Summer Jobs Service, and 
Literacy and Basic Skills. Employers are the primary deliverers of apprenticeship 
training, while community colleges and private training institutions funded by the 
Ministry and the federal government deliver in-school training assistance. 


2. AUDIT OBJECTIVES AND SCOPE 


The objectives of the audit were to assess whether the Ministry had adequate 
systems and procedures in place to: 


e measure and report on the effectiveness of the Division’s major programs in 
meeting stated objectives; 


e ensure that the Ministry and its delivery agencies are meeting labour market and 
participant needs in an economic and efficient manner; and 


e ensure key activities and programs are delivered in accordance with legislative 
and policy requirements. 


The audit focused on four programs: Job Connect; Summer Jobs Service; 
Apprenticeship; and Literacy and Basic Skills. Together they accounted for 75% of 
the Division’s expenditures. The audit was conducted during the period October 
2001 to April 2002, with an emphasis on activities and expenditures in the 2000/01 
and 2001/02 fiscal years.” 


2.1 Overall Audit Conclusions 


The Ministry had taken steps to design systems for providing meaningful 
performance information about its major training programs and the delivery 
agencies it funds. However, the systems and processes necessary to ensure that 
services were delivered economically and efficiently, and that the programs’ 
objectives and expected outcomes were being achieved, had not been completed. 
Successful implementation of current initiatives and the Auditor’s 





recommendations were deemed essential to ensure that the needed systems and 
processes are in place. 


The audit also found that the Ministry had not adequately controlled the acquisition 
and management of consulting and other services acquired through agencies on its 
behalf over the last several years.° 


3. JOB CONNECT AND SUMMER JOBS SERVICE PROGRAMS 


Job Connect was delivered at 123 sites across the province at the time of the audit. 
The 78 agencies delivering the program were community colleges and not-for-profit 
organizations. The services provided were information and referral, employment 
planning and preparation, job development placement supports, and subsidized 
placements. The Ministry has a target of 127,000 clients for this program and has 
allotted it an annual budget of nearly $100 million. 


Summer Jobs Service is offered by most of the same agencies delivering Job 
Connect. It provides a $2 per hour training subsidy to employers hiring students for 
up to 16 weeks. The program’s budget for 2003 is approximately $53 million. It is 
expected to assist about 60,000 youth.’ 


3.1 Measuring and Reporting on Program Effectiveness 


The audit concluded that substantial progress had been made in ensuring that 
funding provided under Job Connect complied with the Management Board of 
Cabinet Directive on Transfer Payment Accountability. On April 1, 2000, the 
Ministry introduced a Continuous Improvement Performance Management System 
(CIPMS) for evaluating delivery agency performance. Six core performance 
measures were established as part of CIPMS. Three of these measures form part of 
a provincial service delivery standard. Failure to meet this standard requires plans 
for improvement within defined timelines. Failure to meet a timeline can result in 
funding being terminated. 


While CIPMS can assist with managing Job Connect effectively, the Auditor 
warned that there is a risk that agencies will report expected results as opposed to 
actual performance, given the funding consequences if the delivery standard is not 
met. He recommended establishing procedures to periodically verify the reliability 
of performance information and completing the development of benchmarking for 
Job Connect to allow comparisons with other jurisdictions. 


In its response, the Ministry said that the statistics reported by agencies were 
validated through cross-references in its database system. Follow-ups were 
conducted where checks showed anomalous data. The Ministry reported that a 
benchmarking process was to start in 2003/04, the year it anticipates all 
performance measures and standards will be fully implemented. 


The sole performance measure established for Summer Jobs Service was a 
comparison of actual to planned intake targets for subsidized placements. The 





Auditor recommended establishing more meaningful measures for assessing agency 
performance and overall program effectiveness. The Ministry said it would factor 
this recommendation into a program evaluation in 2002/03. 


Committee Hearings 


Site visit procedures have been introduced and are being piloted. The Ministry plans 
to implement a systematic data verification and compliance process with transfer 
payment agencies. Pilot testing has begun, with full implementation expected by 
April 2004. The Ministry also hopes to complete a system-wide verification of 
customer satisfaction and employment outcome data, and an analysis of data 
collection processes, by March 2004.” 


3.2 Monitoring Compliance with Program Guidelines 


Program guidelines were revised in June 2001. They describe objectives and 
policies, and indicate the documentation and verification required for participants’ 
files. The delivery agencies visited for the purposes of the audit were found to be 
complying with Ministry guidelines. The annual visits to agencies made by 
Ministry field consultants, however, did not check whether procedures and files met 
Ministry expectations or if reported performance information was accurate. 


The Auditor recommended establishing a risk-based program of periodic visits that 
would help to ensure agencies comply with guidelines and that the performance 
information on which funding is based is reliable. The Ministry stated that it 
conducted visits to agencies below performance standards to assess risk and 
compliance, and to establish improvement targets. A comprehensive review process 
was undertaken for all sites. Other indicators were cited as generating visits and 
reviews as well. All sites would be visited at least once over the three years of a 
funding contract.'° 


Committee Hearings 


As previously mentioned, the Ministry plans to implement systemic data 
verification and a guidelines compliance process which will address the issue of 
monitoring and compliance with program guidelines." 


4. APPRENTICESHIP PROGRAM 


Two acts, the Trades Qualification and Apprenticeship Act and the Apprenticeship 
and Certification Act, govern trades in the motive power, construction, 
industrial/manufacturing and service sectors. They establish specific requirements 
for apprenticeship completion and stipulate that certain trades can only be practiced 
by registered apprentices or holders of a certificate of qualification. Certification is 
optional for all other trades. 


Apprenticeship combines on-the-job training (approximately 90%) with in-school 
training (approximately 10%). The length of an apprenticeship can vary from two to 





five years. An apprentice signs a training contract with an employer. When the . 
signed contract is filed with the Ministry, the apprentice is officially registered. 
Completion of requirements leads to a certificate of apprenticeship. Some trades 
require passing a trade-specific examination to obtain a certificate of qualification. 
Individuals who are not registered can take a trade examination under certain 
circumstances. 


The program is delivered through 26 field offices. Expenditures in 2001/02 
included $58 million to delivery agents for in-school training and for updating 
programs, and $22.9 million for the Ministry’s direct operating costs. There are a 
total of 136 skilled trades. Sixteen new skilled occupations have been designated 
over the past three years in the areas of automotive, service, construction, and 
industrial/manufacturing. Forty-one of the trades are known as red seal trades, a 
designation which allows practise across Canada. At the time of the hearings, about 
55,000 apprentices, 24,000 employers and 50 training agencies were involved in the 
program. ~ 


4.1 Measuring Program Effectiveness 


In June 1998 the Minister announced a commitment to increase the number of 
annual registrations to 22,000 from the 1998/99 level of 11,000. According to the 
Auditor, the Ministry’s annual business plan reports on these registrations but 
contains no other meaningful performance information. Two internal commitments 
were made to increasing the number of active and new employers, and increasing 
the proportion of apprentices who successfully complete their classroom training. 


The Auditor concluded more information was needed to assess the program’s 
contribution to meeting the demand for skilled workers. When combined with 
participant survey information, this data would provide a better indication of the 
appropriateness of the training and of areas requiring improvement. The Auditor 
went on to recommend that the Ministry ensure that the systems then in 
development allow for public reporting of achievements with respect to completion 
and employment rates, and the extent to which the program was meeting the 
expectations of apprentices and employers. The Ministry reported that it was on 
target to implement outcome-based performance measures by January 2004 and 
would then be able to report publicly on achievements, including apprenticeship 
completion and employment rates.'° 


Committee Hearings 


Development of a customer survey has been completed. It will be conducted in 
2003/04. Its results will be used for benchmarking purposes. Work has begun on a 
CIPMS for apprenticeship, similar to that mentioned in section 3.1. The Ministry 
hopes to have it completed by June 2004 and will then report on its achievements. “ 


4.2 Increasing Opportunities for Apprenticeships 


At the time of the audit, a new policy on developing apprenticeship occupations had 
not been finalized. Although 29 new apprenticeship trades had been added over the 


previous four years, they accounted for about 6% of total registrations in each of 
those years. Many of the new trades introduced in the previous two or three years, 
and trades under development, were in traditional areas. In order to meet expansion 
targets, the Auditor maintained that more programs had to be developed in new 
occupations and sectors.'° 


Committee Hearings 


Part of the Ministry’s apprenticeship strategy is the introduction of new 
apprenticeship trades. Work has begun on two: construction craft worker and 
welder. The former has been announced and a regulation is in place. The Ministry is 
working on the training standard. Registration will begin as soon as in-school 
training is in place. In the case of welding, training delivery agencies are being 
identified. 


At the time of the hearings, there were between 16,000 and 17,000 annual 
registrations, an increase of 45% to 50% over the previous four years. Ministry staff 
envisaged reaching the 22,000 level within the next three years. 


The service sector and higher-technology were two areas thought to have 
apprenticeship potential. Reference was made to the success of the Ontario Youth 
Apprenticeship Program. At the same time though, Ministry staff spoke of the 
negative connotations attached to apprenticeship and the need to work with 
industry, unions and employer groups to promote trades work.'® 


Supplementary Information 


Since the hearings, the Committee has learned that the 2003/04 forecast for 
registrations in the construction craft trade is 100." 


Training and curriculum standards for the construction craft trade have been 
approved and are in distribution to apprentices, employers and training delivery 
agents. Training is being delivered by locals and training centres of the Labourers 
International Union of North America and the Universal Workers Union in the 
GTA, Cobourg, London, and Sarnia. Toronto’s George Brown College is 
tentatively scheduled to begin offering training in January 2004. The Minister has 
appointed an industry committee under the Apprenticeship and Certification Act to 
advise on matters related to training and certification. At the time of writing, there 
were about 240 active apprentices in the trade. 


The welder trade was formally introduced in January 2003. Training and curriculum 
standards have been developed and approved by industry, training delivery agents 
and the Ministry. They are in distribution. Training is available from seven 
community colleges and Boilermakers Union locals in Hamilton, Sarnia and 
Toronto. An industry advisory committee is being formed. About 50 apprentices 
were actively engaged in the trade at the time of writing. 





4.3 Updating Apprenticeship Standards 


While the audit found that training standards, curricula, and examinations had been 
updated for most trades with significant apprenticeship enrolment, a few were 
decades old.'’ The Auditor recommended that the Ministry ensure that all training 
standards and examinations are up-to-date and reflect current workplace demands 
as soon as possible.”” 


Committee Hearings 


Exams will be updated in 2003, beginning with the oldest versions. The first was 
replaced in January 2003. At the time of the hearings, it was ready for consideration 
and approval by the provincial advisory committee. Two exams and one training 
standard are under development and will be available for approval in the first 
quarter of 2003/04. Four additional exams and an additional training standard will 
be available for approval in the third quarter of 2003/04. The replacement process 
will be fully completed by April 2004.”' 


4.4 Monitoring Program Quality and Compliance 


The quality of apprenticeship training provided by employers and in-school training 
providers was not being monitored, according to the Auditor. An apprenticeship 
information system was expected to be in place by 2004. Until then, the information 
needed to link funding for in-school training providers to their performance in 
providing quality training or to report publicly on program outcomes will not be 
available. 


The Auditor recommended on-site visits by field staff to employers and training 
providers with identified performance problems, as well as tracking the extent and 
results of those visits. According to the Ministry, field staff do some monitoring on 
their visits. However, the new apprenticeship information system will have features 
that will enhance monitoring activities and assist with workload management. A 
more comprehensive and risk-based monitoring policy and process will be 
developed, within existing resources, to achieve the maximum effect of the visits 
and the capabilities of the new system.”” 


Committee Hearings 


The information system is still in development. An interim internal audit resulted in 
positive feedback. Implementation is scheduled for the second quarter of 2003/04. 
Initial discussions with the Ministry of Labour have indicated that the risk 
management framework for on-site visits being developed by that ministry’s 
Inspections, Investigations and Enforcement Secretariat will help with the system as 
well.” 


4.5 Enforcement of Legislation on Restricted Trades 


The audit found that efforts to co-ordinate enforcement responsibilities and share 
information with the Ministry of Labour and other bodies responsible for workplace 


inspections had not been sufficient to determine the extent to which uncertified 
individuals were working in restricted trades. Effective enforcement was considered 
necessary to ensure legislated objectives for protecting public and workplace safety 
are met and to maintain the value of obtaining certification in restricted trades. 


The Auditor recommended establishing information-sharing protocols, training 
field staff on enforcement expectations, using the new information system to help 
focus enforcement efforts on areas of greatest risk, and monitoring the impact of 
enforcement activities on results. The Ministry agreed that there was value in 
establishing a protocol with the Ministry of Labour. It hoped to begin negotiations 
for a protocol with the Ministry of Transportation, which enforces programs for air 
brake adjustment and truck-tire maintenance, in the fall of 2002. Current guidelines 
and practices would be reviewed and amended as required by March 2003. Field 
staff would receive training regarding enforcement responsibilities in 2003/04." 


Committee Hearings 


Work has begun on developing an enforcement and information-sharing protocol 
with the Ministry of Labour. An initial meeting was held in early February 2003. 
The Ministry hoped to complete the process in March 2003. 


The new information system will be used to help focus enforcement efforts on 
industries, trades and workplaces where a risk has been identified. It will make 
controlled access available through real-time responses to enforcement-related 
queries. Discussions have begun with the Ministry of Consumer and Business 
Services (MCBS), and the Electrical Safety Authority, over which MCBS has 
ministerial responsibility, to determine the extent of Training, Colleges and 
Universities’ involvement with enforcement activities in the electrical trades.” 


Committee Recommendation 
The Committee recommends that: 


1. In strengthening its enforcement and inspection process, the Ministry 
should review current legislation to ensure it allows flexibility for 
potential apprentices to gain some initial experience in a trade without 
registering. 


4.6 Effectively Assessing Prior Learning and Academic 
Readiness 


4.6.1 Prior Learning 


Prior learning assessment identifies and recognizes what a person knows and can 
do. Apprentices and non-apprentices who have the entry requirements for a trade 
can write exemption tests. Only 20 of the 92 trades that require in-school training 
had these tests at the time of the audit. Field offices administered tests for 7 of the 
20, while colleges offered the remaining 13. The audit found that there was no 
formal monitoring of any tests in order to ensure they were being used effectively.”° 





4.6.2 Academic Readiness 


Academic readiness assists apprentices who need help preparing for in-school 
training. The Ministry has developed the Evaluating Academic Readiness for 
Apprenticeship Training (EARAT) tool to identify areas of academic weakness that 
need to be addressed prior to in-school training. The audit found that the Ministry 
had not provided field offices with direction on the use, distribution or 
administration of EARAT products, and did not formally monitor their use.*’ 


The Auditor recommended that exemption tests for each trade be standardized and 
made available. He also spoke of the need to develop and communicate a standard 
policy on the use of these tools, and to monitor the use and results of exemption 
tests and EARAT. The Ministry reported that policies for the use of exemption tests 
and EARAT were in place. Field staff had received training on the use of EARAT 
tools. EARAT general policy was being reviewed to encourage more consistent and 
wider product use. Ministry-administered exemption tests were being tracked; those 
that were flawed would be revised.”* 


Committee Hearings 


All existing trades with an in-school component will have standardized Ministry- 
approved exemption tests by the end of 2003/04. Twenty-two tests have been 
developed and will be validated in the fourth quarter of 2002/03. Updates will be 
ongoing. The Ministry has an interim policy on exemption tests. A full 
communications strategy on that policy will be developed in the second quarter of 
2003/04. Full implementation at Ministry field offices and training delivery agents 
is anticipated in the fourth quarter of 2003/04.” 


4.7 Managing Program Funding for In-School Training 


In-school training received $58 million in 2000/01, 90% of which went to colleges. 
College funding for full-time students is based on planned attendance; that for part- 
time students is based on actual attendance. Private training deliverers received the 
remaining 10% of funding. It is based on actual attendance. A daily per diem is 
used to determine each delivery agent’s entitlement. 


Delivery agents are required to submit audited financial statements to assure the 
Ministry that the per diem rates paid reflect actual daily training costs. Staff from 
several colleges indicated that the statements did not sufficiently reflect daily 
delivery costs. Many statements were found to be overdue and those that had been 
received were not being reviewed. 


The Auditor felt that as the Ministry introduces more outcome-based performance 
measures, ways must be found to link funding to the quality and outcomes of the 
training provided. He recommended working with providers to develop reporting 
that reflects the actual cost of delivery, and introducing funding linked to the 
provision of training that results in positive outcomes for apprentices and 
employers.” 


10 


Committee Hearings 


In August 2002, the Ministry introduced a new funding model that included the 
introduction of classroom fees for apprentices. A final evaluation of fees will be 
completed in 2003/04. It will include a review of the in-school training and delivery 
funding model. Part of the CIPMS planned for the apprenticeship program is key 
performance indicators, one of which will allow for the evaluation of pass-fail 
results by trade, class, level of schooling and by funded training delivery agent. The 
Ministry hopes to have it completed by June 1, 2004. 


Ministry staff were asked how much had been collected from fees. Because the 
process of registering students was ongoing, they estimated $4 million to $5 million 
for 2002/03. Some growth was expected in the following years. Members were also 
told that the Ministry had government approval to reinvest the money collected in 
the training system, including apprenticeship. 


When asked if enrolment would be tracked to determine if fees were a barrier, 
Ministry staff reported that enrolment was tracked on an annual basis and that the 
impact of fees will be monitored. Other provinces with fees had indicated that there 
is a brief drop-off and then the numbers start to rise. Thought would be given to 
tracking this information through the use of surveys.* 


Committee Recommendation 
The Committee recommends that: 


2. The Ministry report to the Committee on the results of its evaluation 
of classroom fees, and its review of the in-school training and delivery 
funding model, by June 1, 2004. 


5. LITERACY AND BASIC SKILLS PROGRAM 


The Literacy and Basic Skills Program serves about 200,000 clients: 40,000 in 
intensive language services and 160,000 in training orientation services. The budget 
in 2002/03 was about $60 million. At the time of the audit, the Program supported 
and funded 200 literacy agencies, school boards and community colleges providing 
services at approximately 300 sites across the province. About 10,000 volunteers 
help with the program. The Ministry has also developed AlphaRoute, a web-based 
learning tool to facilitate the acquisition of literacy and numeracy skills. 


Efforts in the area of literacy and basic skills take on extra relevance in light of the 
findings of the most recent /nternational Adult Literacy Survey. That 1994 
publication reported that approximately 20% of adult Ontarians did not have the 
basic literacy skills to meet workplace and daily living requirements.” fe 
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5.1 Tracking and Reporting Participant Outcomes 


A performance measure for tracking outcomes for participants was established in 
1999/00: the percentage of participants who got a job or went on to further 
education or training, as measured three months after completing or leaving the 
program. The Ministry’s business plan for the following year stated a commitment 
to having delivery agencies provide 65,000 clients with initial services. More 
intensive services were to be received by 68% of that total. Seventy percent of those 
receiving more intensive services were to obtain employment or pursue further 
education or training. 


The 2001/02 business plan increased the target numbers but did not report on 
2000/01 results. Actual accomplishments were reported internally as 62,800 
participants served, 80% of whom got a job or went on to further education or 
training. The Auditor considered these results misleading for a series of reasons. 
For example, agencies did not use consistent practices in determining which clients 
to survey and they were unable to contact about 28% of the clients they tried to 
survey. The Ministry was also unable to track the length of time clients stayed in 
the program. 


The Auditor recommended ensuring that delivery agencies consistently conduct and 
report the results of participant outcome surveys, requiring that all agencies take 
steps to minimize lost contacts and report them as part of program performance, 
tracking and reporting the length of a client’s time in the program, and reporting 
actual performance results in its business plan.*° 


Committee Hearings 


Steps have been taken to ensure that all agencies are consistently conducting and 
reporting on follow-ups. Guidelines have been developed and are posted on the web 
site. Lost contacts will be included in the calculation. The information management 
system allows the Ministry to generate a report that tracks the length of time clients 
remain in the program, which will help in the development of performance 
measures." 


5.2 Linking Funding to Performance 


The number of contact hours provided by a delivery agent is negotiated annually. 
Funding ranges are based on the negotiated level of contact hours. The Auditor felt 
there was a need to establish a link between funding levels, and the quality and 
effectiveness of services. He also believed that the level of funding was not 
sufficiently responsive to changes in activity levels, in total or at individual 
agencies. 


The audit reviewed the funding and reported activity at a sample of sites for 
2000/01. It found that 37% of the sites only provided between 52% and 83% of 
approved contact hours but spent virtually all of the Ministry funds provided. 
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The Auditor recommended implementation of a funding model that sets out the 
conditions and process which will result in adjustments in funding, and recognizes 
an agency’s success in helping clients achieve position outcomes. According to the 
Ministry, the funding model was being reviewed, based on a principle of 
transparency and linked to performance standards. It will establish benchmarks and 
adjust funding levels based on performance and circumstances.* 


Committee Hearings 


A new funding model is still in development and will hopefully respond to the 
Auditor’s recommendation. The Ministry hopes to see recommendations for the 
model in the late spring of 2003 and also hopes that they include a phased-in 
approach that will begin in November 2003, to start up in 2004/05.*° 


Committee Recommendation 


The Committee notes that a funding model for the Literacy and Basic Skills 
Program was still in development at the time of the hearings. 


The Committee therefore recommends that: 


3. The Ministry report to the Committee no later than December 31, 
2004, on how that model will respond to the Auditor’s recommendation 
linking funding to performance. 


5.3 Monitoring Delivery Agency Performance 


Field consultants are required to visit each agency approximately every 12 to 18 
months. They base their determination of which agencies to visit on a series of 
factors, including the time of the last visit and overall impressions of the agency’s 
success. The audit found that the program for visits needed to be strengthened 
because of the lack of a formal assessment of risk factors and sufficient oversight of 
the process. An assessment tool was being developed at the time of the audit. The 
Auditor recommended ensuring that consultants formally assess the risk of 
performance problems when selecting and conducting visits, and the tracking and 
summarizing of the results of all monitoring visits.” 

Committee Hearings 





Work has begun on formalizing a risk-assessment component for use during agency 
visits. A tracking system will be completed to ensure that the timing of all agency 
visits complies with the risk-assessment approach. Corrective action will be taken 
as required. Rating assessments will begin in April 2003. The Ministry hopes to 
have them completed by March 2005.** 
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6. ACQUISITION AND MANAGEMENT OF CONSULTING AND OTHER 
SERVICES 


The Auditor noted that since 1998/99, the Ministry had paid two not-for-profit 
agencies approximately $11 million to undertake projects and to purchase related 
consulting, information technology (IT), and other services for the administration of 
the Job Connect, and Literacy and Basic Skills programs. In so doing, it had not 
followed prudent purchasing practices and had not obtained the approvals from the 
Minister and Management Board of Cabinet that would have been required had the 
projects been undertaken and the services acquired by the Ministry directly. The 
Ministry had also paid about $600,000 in GST on these services. It would not have 
had to pay this had it engaged providers directly. 


The Auditor had concerns about the ways the services were acquired, funds 
advanced and contracts managed. He felt the Ministry could not demonstrate that 
value for money was obtained for the services provided. His recommendation called 
on the Ministry to comply with government policies for the acquisition of such 
services, regardless of the source of funds used to acquire them, and to recover any 
funding provided to transfer payment agencies related to GST they were not 
required to pay.*’ The Auditor did acknowledge that the Ministry had committed to 
improving and had already “taken some action to strengthen its procedures for 
managing all existing and future projects.” 


Committee Hearings 


Procedures for managing all existing systems and consulting contracts have been 
strengthened. Development of the new apprenticeship IT system began in 2001 and 
was fully tendered. Companies were selected through an RFP process verified by an 
internal audit. Prudent controls and processes were incorporated into the acquisition 
of IT services for 2002/03. A Ministry-wide risk assessment project will allow for 
the identification and mitigation of risks during the procurement process. 


Future systems development for RFPs will be issued through MERX, the public 
sector electronic tendering service, or through an invitational bid of companies on 
the vendors-of-record list. Mandatory procurement training for all managers will 
take place in the near future." 


6.1 Information Systems Development Projects 
6.1.1 Job Connect 


A third-party, not-for-profit agency was asked to co-ordinate the development and 
implementation of Job Connect’s information system in 1998/99. A company was 
hired without a competitive process by the agency to co-ordinate the project. The 
company said it hired a systems development firm through a selection process, but 
audit staff were unable to review this process “because the company was not 
required to retain the relevant documentation for more than three years.” From 
1998/99 to 2001/02, the agency paid the company $4.4 million in Ministry transfer 
payment funds. Included in the total were billings from the systems development 
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firm. The Ministry’s annual funding also included money for functional 
enhancements, system maintenance and delivery agency training. 


The Ministry requested annual proposals including planned deliverables and 
corresponding costs from the company between 1999/00 and 2001/02. The annual 
agreements with the agency itemized deliverables to be provided by the company 
but not the corresponding costs, which were approved as a lump-sum. The Ministry 
had requested annual audited financial statements. They provided assurance on the 
total project expenditures incurred by the company but not on costs incurred for 
each deliverable.” 


Committee Hearings 


The Job Connect system is operated in partnership with the Ontario Association of 
Youth Employment Centres (OA YEC). The OAYEC retained IT consultants to 
handle the technical aspects. Different IT consultants have been involved with the 
project since its inception. 


Ministry staff acknowledged that they should have more fully scoped the project, in 
terms of the work required, in its initial stages. Subsequent to the audit period, 
further system development work was tendered. Help desk support will be tendered 
when the current agreement ends next year. The Committee was given a 
commitment that the Ministry would look at more in-house support.” 


6.1.2 Literacy and Basic Skills 


A not-for-profit agency was asked to lead the development of Literacy and Basic 
Skills’ information management system in March 2000. The agency also served as 
project manager. In August 2000, it selected a consulting firm to develop the system 
based on a bid price of $700,000. Three months later, the agency submitted a more 
comprehensive project proposal with costs totalling $3.8 million. The system 
development component had increased from $700,000 to approximately $1.6 
million. The Ministry approved the $3.8 million proposal in December 2000. 


The system was not complete by March 31, 2002, even though total costs billed had 
reached $3.6 million. According to the Auditor, development costs had increased 
due to three factors: project deliverables had not been finalized before the RFP was 
issued in August 2000; the Ministry had not directed the project manager to include 
the second phase in the first RFP, eliminating any possible economies of scale; and 
the system development consultant was able to add 35% to its original bid to allow 
for unanticipated changes. As with Job Connect, the audited statements provided 
the total costs incurred by the agency only.” 


Committee Hearings 


Ministry staff acknowledged that all aspects of the project should have been scoped 
before going to tender. They had wanted to work with the literacy community on 
the design and also wanted a system that would allow for a better response to the 
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Auditor in terms of monitoring performance, knowing who delivered good service 
and knowing more about outcomes. 


The Ministry’s sense was that it had purchased “a fairly good system’ and did not 
overpay. Another consultant would have to be hired, if an independent assessment 
was actually required. 


The Literacy and Basic Skills system is up and running. The Ministry now receives 
monthly reports, as opposed to semi-annually, from 200 delivery agents at 300 sites. 
It will also have a much better performance management system, as performance 
will be tied to funding. 


There were contracts in 2002/03 to do further work on development, even though 
the system is up and running. There was a tendered contract this year to develop the 
reporting structure for more data to support better monitoring of performance, 
ensure value for money and improve client service. In addition, an IT firm operates 
the help desk. At the time of the hearings, the help desk service was about to be 
tendered. Once that contract ends in 2004, the Ministry will decide whether the 
work could be performed for less within government. 


Following on this commitment to consider going in-house for IT support in the 
future, Ministry staff commented on the generic nature of help lines services. At the 
same time though, they told the Committee of early concerns about the availability 
of IT resources within government and reminded Members that this technical 
support was being provided to transfer payment agency users, for the most part, 
rather than Ministry users.*° 


Supplementary Information 


During the hearings, the Committee was told that the Ministry would provide data 
on the value of contracts for further work on system development in the 2002/03 
fiscal year. The Committee has since learned that the 2002/03 funding allocation of 
the Literacy and Basic Skills information management system was $1.9 million.”’ 


Committee Recommendation 


The Committee appreciates the actions taken by the Ministry to strengthen 
procedures for managing information systems and consulting contracts. However, it 
continues to be concerned about the activities of third-party agencies. 


The Committee therefore recommends that: 


4. The Ministry ensure that all third-party agencies are fully aware of 
and follow prudent purchasing and project management practices as is 
required of the Ministry by Management Board of Cabinet, and that it 
report to the Committee within one hundred and twenty calendar days 
of the date of tabling this document with the Speaker, on the Literacy 
and Basic Skills system’s costs and effectiveness. 


16 


6.2 GST Payments 


The audit found that because the consultants had billed the agencies for services 
rendered, they included GST. The Ministry is GST-exempt. Because it used not-for- 
profit agencies, it paid approximately $600,000 more for these services than it 
would have had it contracted directly with the consultants. The Ministry reported 
that it had already taken steps to recover GST that the agencies were not required to 
pay. It would also ensure that GST was appropriately funded in future projects of 
similar scope and size, and explore the implications of and options for developing 
polictes/procedures related to the recovery of GST spent in the delivery of program 
services. 


Committee Hearings 


The Ministry adjusted contracts in 2002/03 and has started the recovery process.” 


7. ADDITIONAL POINTS OF DISCUSSION 


Members’ discussions with Ministry staff ranged beyond the general purview of the 
audit to fully embrace the Training Division’s jurisdiction and that of the Ministry 
of Training, Colleges and Universities. The topics considered below appear in the 
order in which they were introduced. 


7.1 Labour Market Development Agreement 


Ontario is the only province or territory in Canada without a labour market 
development agreement (LMDA) with the federal government. Ministry staff told 
the Committee that this has resulted in weak co-ordination between federal and 
provincial training initiatives, as well as confusion and duplication of overhead and 
administration. Other jurisdictions have been able to completely integrate their 
labour market systems.” 


Human Resources Development Canada (HRDC) first approached the provinces 
and territories with the offer of LMDAs in 1996.°' Two types of agreement were 
available: devolution of federal programming and staff or no devolution, just better 
joint planning. Ontario wanted a full devolution agreement, but that would have 
involved integrating 1,007 federal employees into the Ontario Public Service with 
three-year job guarantees. This was difficult in 1996 as the province was looking at 
staff reductions of its own. Another point of contention was funding. Ontario was 
offered approximately $600 million but considered this to be about $145 million 
less than its fair share, based on the province’s percentage of the country’s 
unemployed. 


HRDC continues to spend money on federal priorities in Ontario. According to 
Ministry staff, this funding has been increasingly spent on promotion and less on in- 
school training. Members were told that two years ago, HRDC did not spend its 
entire Ontario budget. In those jurisdictions with LMDAs, especially devolution 
agreements, federal monies were fully expended. 


1? 





In 2001, Ontario indicated that it would accept the federal terms and conditions 
without amendment (i.e. the funding and accepting federal staff). These outstanding 
Issues were now deemed secondary to improving and integrating Ontario’s training 
system. The federal response was described as not positive. Ministry staff said 
further efforts had stalled because the federal government had introduced its own 
skills agenda in 2002. 


Supplementary Information 


As requested by the Committee, the Ministry supplied copies of selected 
correspondence between the provincial Minister of Training, Colleges and 
Universities and the federal Minister of Human Resources Development respecting 
a federal-provincial LMDA. The correspondence provided covered the period from 
July 20, 2001 to April 11, 2002.°° 


In her letter of April 11, 2002, the federal Minister referred to the time which had 
elapsed between her government’s offer of LMDAs to the provinces and territories, 
and the Ontario government’s request to seek a LMDA on the basis of that original 
offer. She wrote of “significant shifts” in the Canadian labour market since 1996 
(structural changes, emerging skills shortages and a downward trend in 
unemployment). In the context of these changes and governments’ skills and 
learning priorities, she felt it did not make sense to “undertake new bilateral 
negotiations based on an outdated model.” 


7.2 Foreign-Trained Professionals and Tradespeople 


Approximately 60% of immigrants to Canada settle in Ontario. About 235,000 are 
expected to come this year. Most of the adults, 70%, arrive with some post- 
secondary education and/or training.’ 


The employment difficulties faced by foreign-trained professionals and 
tradespeople occupied much of the time the Committee spent in discussion with 
Ministry staff. Comments and questions covered a range of topics such as 
apprenticeship and foreign-trained workers, investments in language training, and 
internal versus external renewal of the labour force. However, many touched on two 
issues Ministry staff described as critical to the successful integration of those 
trained outside Canada: the availability of information and barriers to employment. 


7.2.1 Availability of Information 


Many immigrants arrive in Ontario with a lack of knowledge about the realities of 
the provincial labour market, most particularly the requirements and expectations of 
employers and professional/trades associations. The Committee heard that both the 
provincial and federal governments could do a better job of providing vital 
information to foreign-trained workers prior to leaving their home countries. 
Ontario was said to be addressing the issue through the information sheets on 
individual trades and professions it has mounted and will be mounting on the 
Ministry of Training, Colleges and Universities’ web site. 
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The federal government has been conducting consultations through which it has 
heard of the need to equip immigration counsellors and embassy staff with better 
information. The Committee was told that Ottawa has been talking about an Internet 
portal which would present syntheses of materials from each of the provinces in a 
uniform format.” 


Committee Recommendation 


If the federal government decides to pursue the idea of an Internet portal, the 
Ministry of Training, Colleges and Universities will need to be actively involved in 
its development in order to represent provincial interests. 


The Committee therefore recommends that: 


5. The Ministry encourage the federal government in all efforts that it 
may undertake to develop a national Internet portal for the use of 
foreign-trained professionals and tradespeople in their home countries 
and in Canada. Participating provincial/territorial jurisdictions should 
be given the opportunity to provide input in its development to ensure 
that the initiative, while consistent in its presentation, addresses their 
diversity of needs and interests. 


7.2.2. Barriers to Employment 


Barriers to employment include language, credential assessment, knowledge about 
the Ontario workplace, and licensing requirements. Ministry staff commented on 
their efforts to develop bridging programs in areas such as nursing, engineering, 
teaching, and skilled trades. These programs can incorporate assistance with 
language training, information about the credential assessment process, and a work 
term. 


A group of regulatory bodies has been working with the Ministry on the preparation 
of tools and materials for other regulators to raise their awareness and encourage the 
development of relevant initiatives. While these bodies are justifiably concerned 
about standards, census predictions about the labour supply and the public attention 
being paid to skill shortages have resulted in more active participation, on their part, 
in finding ways to break down barriers.°° 


Supplementary Information 


The Ministry of Training, Colleges and Universities is working with twelve 
occupational regulatory bodies in the areas of accountancy, architecture, health, 
engineering, and education on Access to Professions and Trades initiatives.” y 


7.3 Local Training Boards 


Local training boards fall under the Training Division. The province spends about 
$3 million a year supporting their work on local labour market planning and 
information, environmental scans and assessments of local labour market 
conditions. Boards also work with local partners to strengthen the local labour 


19 





supply. Human Resources Development Canada partners with the province and 
supports boards with about the same amount of money as the province.” 


Supplementary Information 


There are 21 local training boards located across the province.”” 


7.4 Double Cohort 


The so-called double cohort will be entering the province’s post-secondary 
educational institutions in the 2003/04 academic year. Ministry staff enumerated the 
various measures the government has put in place (e.g. capital investments, 
operating funding, increased student awards) to handle the influx but acknowledged 
that more students were applying than they had expected. They also pointed out that 
Training Division activities (apprenticeship and Job Connect) will be affected since 
50% of high school graduates do not pursue post-secondary education; they enter 
the job market. 


Students in the double cohort will be graduating as members of the Baby Boom are 
retiring, adding to the available labour pool. 


Colleges and universities in Northern Ontario have experienced increases in 
applications for the upcoming academic year. Several of them were said to be 
having difficulties with enrolment, but Ministry staff referred to the province’s 
recognition of the increased costs they faced due to distance and smaller 
campuses.” 


Supplementary Information 


The average number of applications to Ontario universities made by Ontario 
secondary school students in 2003/04 is 5.07. The figure for 2002/03 was 4. 16.°! 


7.5 Northern Ontario Medical School 


The first meeting of the board of the new Northern Ontario Medical School was 
held several weeks before the Committee’s February 2003 hearings. The School 
will have two campuses; one in Sudbury with 32 students and the other in Thunder 
Bay with a class of 24. The expected starting date is September 2004. Work has 
commenced on designing curriculum and promoting the School, and working with 
local hospitals and medical associations. 


In their third and fourth years, students will practise in Sudbury and Thunder Bay, 
as well as in other northern communities. Ministry staff made reference to studies 
which indicate that if someone is educated in Northern Ontario, there is a greater 
chance of keeping them there once they are permanently employable. 


The School has yet to decide on an admissions policy. It is engaged in a debate 
about the treatment of students from Northern Ontario and those from the south. 
There is also interest in programs for francophone and Aboriginal students who are 
important components of Northern Ontario's population.” 


20 


Supplementary Information 


During the hearings, the Ministry was asked about incentives for new doctors to 
practice in Northern Ontario. The Committee has since been told that the Ministry 
of Health and Long-Term Care provides $4 million each year for tuition and 
location incentives to new doctors willing to practice in underserviced areas of the 
province, including Northern Ontario. Medical students are entitled to receive 
$10,000 for each year of undergraduate medical training as part of the Tuition Grant 
Program and Location Incentives fund. In exchange, the students agree to practice 
medicine in a designated underserviced area for a minimum of three years.” 


7.6 E.S.L. Funding 


The Ministry of Education and the federal government both provide funding to 
E.S.L. (English as a second language) programs. Federal money flows through a 
program called LINC (Language Instruction for Newcomers to Canada). According 
to Ministry of Training, Colleges and Universities staff, Ontario could make a case 
for more federal money given the proportion of immigrants coming to this province. 
Nonetheless, they were optimistic about the then upcoming federal budget and 
recent federal “signals” of interest in more higher level and workplace language 
training. 


Supplementary Information 


The LINC program focuses on basic or lower level language training to support 
new immigrants through the settlement process. The 2003 federal budget included 
$10 million over the next two years to pilot advanced labour market language 
training across the country.® 


8. LIST OF COMMITTEE RECOMMENDATIONS 


1. In strengthening its enforcement and inspection process, the Ministry should 
review current legislation to ensure it allows flexibility for potential apprentices 
to gain some initial experience in a trade without registering. 


2. The Ministry report to the Committee on the results of its evaluation of 
classroom fees, and its review of the in-school training and delivery model by 
June 1, 2004. 


3. The Ministry report to the Committee no later than December 31, 2004, on how 
that model will respond to the Auditor’s recommendation linking funding to 
performance. 


4. The Ministry ensure that all third-party agencies are fully aware of and follow 
prudent purchasing and project management practices as is required of the 
Ministry by Management Board of Cabinet, and that it report to the Committee 
within one hundred and twenty days of the date of tabling this document with 
the Speaker, on the Literacy and Basic Skills system’s costs and effectiveness. 
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5. The Ministry encourage the federal government in all efforts it may undertake 
to develop a national Internet portal for the use of foreign-trained professionals 
and tradepeople in their home countries and in Canada. Participating 
provincial/territorial jurisdictions should be given the opportunity to provide 
input in its development to ensure that the initiative, while consistent in its 
presentation, addresses their diversity of needs and interests. 
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